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ABSTRACT 



A project was conducted to establish criteria by which adult 
basic and literacy education (ABLE) student enrollment, completion, early 
separation, and educational level/category attainment are reported throughout 
Pennsylvania. First, staff at 10 Project Educational Quality for Adult 
Literacy (EQuAL) sites were interviewed to identify recurring issues and 
patterns in the reporting of student data. Next, federal and state 
data- reporting requirements were compared and reconciled with those used at 
EQUAL sites. A working group of adult education professionals and ABLE 
personnel developed draft criteria for data reporting. The criteria were 
finalized, a data dictionary was developed, and training guidelines and plans 
were developed to train ABLE program staff in the new criteria for data 
reporting. Definitions were developed for the concepts of enrollment, 
completion, continuation, and early separation, and it was decided that the 
federal definitions of instructional categories for the following categories 
of students would be adopted by ABLE: beginning literacy, beginning English 
as a second language (ESL) ; intermediate ESL; and advanced ESL. (Appended are 
the following: the federal definitions; training plans; and codes and 
descriptions of the data fields in the new data dictionary.) (MN) 
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ABSTRACT 



Grant Recipient: Center for Literacy 
636 South 48* Street 
Philadelphia, PA 19143 
Phone: (215) 474-1235 

Project Name: Development of Criteria for Student Data Reporting 

Grant Allocation: $ 19,783 

Project Number: 98-7002 

Project Period: July 1, 1996 - June 30, 1997 

Project Director: Ashley Stoudt 

Contact: JoAnn Weinberger 



Project Purpose: The Development of Criteria for Student Data Rq)orting project investigated 
recurring issues and patterns in the reporting of student data in order to estabhsh the criteria by 
which student enrollment, completion, early separation, and educational level/category 
attainment are reported statewide. EQuAL Project findings as reported by ten participating 
programs revealed programmatic issues in student data reporting, while Federal and state 
requirements were compared and reconciled. These finding s supported the writing of draft 
criteria, which were then reviewed by a working group of adult ^ucation professionals and 
ABLE personnel. Upon ABLE approval, the criteria were finalized, a data dictionary was 
composed, and training plans were established. Dissemination of the criteria will take place in 
September 1997 via a statewide teleconference in which ABLE- funded programs and 
Professional Development Centers will be trained. 

Project Outcomes: Criteria for student data reporting were developed, resulting in a common set 
of definitions by which both programs and ABLE can interpret data to assess quality of service 
provision and number of students served. 

Impact: The project impact will be evaluated after ABLE-funded programs are trained and begin 
to implement the criteria during program year 1997-98. 

Product or Training Developed: Criteria estabhshed, a data dictionary composed, and training 
plans developed for use in the 1997-98 program year. Training for ABLE-ftmded programs and 
Professional Development Centers will occur via a statewide teleconference in September 1997. 

Products Available from: AdvancE and Western Pennsylvania Adult Literacy Resource Center. 

Project Continuation: The project will continue in the 1997-98 program year with training on the 
criteria occurring in September 1997. 
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FINAL REPORT 



Introduction 

In program year 1996 - 97, the Center for Literacy (CFL) conducted a 353 Special 
Demonstration project to develop criteria for reporting student data by adult basic and literacy 
^ucation providers throughout the Commonwealth. The intent of the project was to estabUsh a 
set of standard criteria by which all programs could define student data regarding enrollment, 
educational progress and attainment, early separation, completion, and continuation. The project 
was conducted by Ashley Stoudt, Program Coordinator for Staff and Program Development at 
CFL. 

Audience 

The audience for this project includes the following three groups: 

At the program level, directors and administrators who are responsible for educational 
service delivery, program evaluation and planning, and staff development; direct service 
providers who are responsible for assessing, placing, and planning instruction based on 
information on student progress; data managers who are responsible for processing and reporting 
student data internally and externally. 

Regional Staff Development Center directors and personnel who are responsible for 
disseminating the criteria and providing training programs in the new criteria for student 
reporting. 

ABLE personnel who will evaluate and approve the criteria for use in programs 
statewide. 

1. Statement of the Problem 

The Commonwealth of Pennsylvania possesses a diverse field of adult basic and literacy 
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education programs that strive to meet the needs of a range of adult learners. Programs differ in 
the population of students served; missions or philosophies; models for service 
delivery/instruction; and the characteristics of both direct service personnel and administration. 

These realities result in multiple conceptions of what constitutes student enrollment, 
progress, completion, early separation, and outcomes; in turn, these diverse conceptions lead to 
development of various processes and methods of collecting, analyzing, reporting, and evaluating 
student data. 

To address these and other issues in adult basic education. Indicators of Program Quality 
were developed by the U.S. Department of Education to establish criteria for student educational 
outcomes, program planning, and instructional and curriculum materials to enable programs to 
monitor their progress and to provide a standard for accountability. The Bureau of Adult Basic 
and Literacy Education, Peimsylvania Department of Education added two Indicators and asked 
each program to develop standards and criteria for the 1994-95 year. The Indicators are guiding 
principles to which programs are held accountable, but they are intended to provide flexibility for 
individual program interpretation and implementation. Indicators are process-oriented, rather 
than outcomes-oriented; thus, they have been integrated into the statewide monitoring process. 

As a result, agencies that are diverse in demographics, mission, and goals have developed 
different measures for evaluating program quality according to the Indicators. 

While it may initially appear adequate and even advantageous for programs to develop 
approaches and methods independently according to their specific contexts, the reality is that at 
the statewide level no reliable and valid criteria exist to be applied across all programs to 
evaluate the reporting of student data. Ultimately, this problem inhibits the overall goal of 
increasing program accountability by limiting any program’s ability to self-monitor or 
comparatively evaluate its performance and outcomes. It also makes impossible the interpretation 
of data submitted to the Commonwealth. 
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A related project is the current 353 funded Special Demonstration Project Educational 
Quality for Adult Literacy (Project 98-6010), begun in 1994-95 and continuing in 1995-96 and 
1996-97, which is a statewide initiative to develop a performance evaluation system. The first 
year report of EQuAL Project outcomes identified the following assessment issues: 

• The criteria for student progress require review. 

• Student outcome data are often not collected because programs usually focus on 
student needs rather than successes. 

• Assessment procedures and mstruments need to be reviewed by programs. 

• English as a Second Language programs may need different assessment instruments. 

• Student self-assessment may increase motivation. 

As these outcomes indicate, the EQuAL Project’s initial findings will inform the development of 
the student data reporting criteria. Likewise, the resulting criteria for reporting student data will 
inform the ongoing efforts of EQuAL. The criteria development, E()uAL, and the competencies 
project will work in tandem with the Bureau of Adult Basic and Literacy Education to ensure 
consistency. Ultimately, the related statewide program quality initiatives will be joined under the 
multi-year efforts of the EQuAL Project. 

2. Goals and Objectives 
The two project goals were: 

1. To develop criteria for adult basic and literacy education providers to use in reporting 
student data to the Bureau of Adult Basic and Literacy Education. 

2. To train ABLE-funded programs and Professional Development Center staff on the new 
criteria to be used for student reporting. 
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Specific objectives were: 



1 . To conduct interviews with ten (10) agencies participating in the EQuAL project to 
determine how staff currently define and record student termination, assessment, and 
outcomes data. 

2 . To review and compare ABLE reporting requirements with Federal requirements, and to 
reconcile discrepancies. 

3 . To submit an issues report to ABLE for review; and to draft criteria based on the 
response. 

4. To convene a working group of adult education professionals to evaluate and finahze the 
criteria for the following: 

• Student completion, advancement, and early separation. 

• Number of levels advanced. 

• Levels completed and when these levels constitute successful completion of 
the program. 

• Obtaining a job or advancing in a job, meeting personal objective, completing 
a level, learning English. 

• Other outcomes included on the student form. 

5. Upon ABLE approval of the final criteria, to train Regional Staff Development Center 
staff on the criteria. 

6. To publish criteria in a usable report, in the form of a data dictionary. 

3. Procedures 

The project investigated recurring issues and patterns in the reporting of student data. The 
methodology for data collection, which involved the interviewing of ten EQuAL program sites 
who have been examining issues regarding student intake, assessment, and data collection, 
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assured that the project acquired an understanding of both the depth and breadth of the issues in 
student data reporting requirements. 

? 

Comparison of the state and Federal student data requirements revealed any ♦ 

discrepancies. Analysis of the program data and reporting requirements then informed the next 
phase of the project, the definition of criteria for data reporting. The project director drafted 
definitions which were reviewed and revised by a working group of adult educators, comprised 
of program-based and research-based professionals in a one-day working session. The resulting 
definitions were submitted to the Bureau of Adult Basic and Literacy Education for approval; the 
data dictionary was produced; and training plans were developed. The project will disseminate 
them to the ABLE-fimded programs and to Professional Development Centers and provide 
training guidelines to support adoption by all programs. 

4. Objectives Met 

The project was able to meet all of its objectives except for the completion of training 
before the end of the project year. 

In accomplishing its objectives, the project interviewed ten programs statewide; All 
programs were participants in the EQuAL Project, and thus had been engaging in an examination 
of program quality issues. The programs chosen to participate represented a range of learner 
populations, numbers served, and regions of the state. Program participants were; Community 
Women’s Education Project, Philadelphia; Goodwill Industries Literacy Program, Pittsburgh; 
Lancaster Lebanon I.U. 13; Midstate Literacy Council, State College; Northwest Tri-County I.U. 

5, Erie; Tri-County OIC, Harrisburg; Tuscarora I.U. 11, Lewistown; Altoona School District; 
Greater Johnstown Career and Technology Center; and Greater Pittsburgh Literacy Council. 

Representatives fi-om each program were interviewed via telephone or email. Interview 
questions were intended to be open-ended, yet to target particular aspects of student data 
reporting in practice. Questions included: 
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What is the setting of your program? WhtTdo you serve? How does a typical intake and 
assessment proceed? 

How and when does your program count a student as enrolled? 

How does your program measure and record student progress? 

How does your program exit students? 

Who gathers and reports your program’s data on students? 

Are there other issues that you see in student data reporting that you would like to share? 

Programs commumcated the issues they found in practice, and this information was 
synthesized with the discrepancies found between state and federal reporting re<iuirements to 
form a working set of issues. These issues were reviewed and addressed with ABLE. 

Based on the core issues that were identified, draft criteria were written for presentation 
to the Review Committee of adult educators, comprised of the following: Pat Haff, Community 
Women’s Education Project; Judith Aaronson, Goodwill Industries; Sandy Strunk, Lancaster 
Lebanon I.U. 13; Monica Mathews, Midstate Literacy Council; Lee Knisley, Tri-County OIC; 
Jean Fleschute, Community Learning Center; Sue Conrady, I.U. 1; Claire Russell, La 
Comumdad Hispana; Susan Clark-Teisher, PA College of Technology; and Joan Breisch, 
Reading/Berks Literacy Council. This review committee suggested revisions to strengthen and 
clarify the draft criteria, and upon ABLE approval, the criteria were successfully finalized. 

A data dictionary was prepared, specifying each aspect of the criteria in relation to the 
existing electronic database. Lastly, training guidelines and plans were developed for the 
teleconference to train programs in the new criteria for student data reporting. 

5. Objectives Not Met 

Due to time restrictions, the project was not able to achieve its final objective to provide 
statewide training for programs within the program year. However, the training is planned for 
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September 1997. 



6. Evaluation 

Informal evaluation occurred as the project Review Committee considered and finalized 
the draft criteria during its May meeting in Harrisburg. The committee generally approved the 
criteria, while suggesting some revisions to improve clarity. 

Final project evaluation will occur after the training teleconference is held and programs 
begin to implement the criteria. A training evaluation will be conducted to gather feedback from 
participating programs. Ultimately, the success of the project will be evaluated by the degree to 
which programs are able to effectively and accurately report data using the criteria, which in turn, 
results in useful program data for ABLE. 

7. Dissemination 

Dissemination of project products will occur in conjunction with the training in September 
1997, and the final criteria, the data dictionary, and training plans are available concurrent with 
the submission of the final report. 

8. Conclusion and Recommendations 

This program-directed and program-based project enabled programs themselves, in 
conjunction with ABLE, to establish the criteria by which student enrollment, completion, early 
separation, and outcomes are reported and what criteria should be used to gather, analyze, and 
report the data. 

Through the reporting of student data according to established criteria, ABLE will have a 
common means by which to summarize data across programs as well as to evaluate the extent to 
which programs are meeting goals both in quality of service provision and numbers of students 
served. 
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Development of Criteria for Student Data Reporting Project 

CRITERIA 



I. ENROLLMENT 

A student will be considered enrolled after receiving 1 2 hours of instruction. The following 
definitions apply: 

A. Initial Assessment: Includes all activities related to initial intake, including 
assessment, orientation, class visits, etc. Up to five initial assessment hoitrs may be 
applied toward a student's accrual of horns. 

B. Instruction: Defined as activities involving direct teaching and ongoing assessment 
which is directly monitored or supervised by educational staff. 

n. INSTRUCTIONAL CATEGORIES 

ABLE will adopt the Federal definitions of instructional categories for students. Federal 
definitions of functioning categories describe a learner’s abilities at entry in basic skills. The 
definitions provide general indicators and include test benchmarks as examples; grade level or 
CASAS are given for ABE; CASAS or SPL/MELT for ESL). See Appendix. 

III. COMPLETION AND CONTINUATION 

New definitions for completion and continuation will be as follows: 

A. Completion: A student has: 

1 . Completed the category at which enrolled and left the program; or 

2. Completed learning goal(s) and left the program. 

B. Continuation: A student has: 

1 . Continued in the program while moving to a higher category; or 

2. Continued in the program after attaining original learning goal in order to attain 
other learning goal(s); or 

3. Continued in the program in the same category at which enrolled or without 
attaining learning goal(s). 

Note: These definitions are intended to enable programs to report students who achieved learning 
goals both <12 and >12 hours of instruction. 

rv. EARLY SEPARATION 

The definition for early separation is as follows: 

A student separated early prior to completing the enrollment category or attaining a 
learning goal. 
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Beginning Literacy 

Functional Skills: The individual functions minimally or not at all in English and can 
communicate only through gestures or a few isolated words. The individual may lack literacy in 
the native language and has had little or no formal schooling. 

Reading and Writing: The individual cannot read or write or can read or write only isolated 
words. There may be little or no alphabet recognition. 

Speaking and Listening: The individual cannot speak or understand English, or understands only 
isolated words or phrases. 

Examples of Test Scores: 165 - 180 CASAS; SPL/MELT 0 -1. 

Beginning ESL 

Functional Skills: The individual functions with difficulty in situations related to immediate 
needs and in limited social situation; has some simple oral communication abilities using simple 
learned and often repeated phrases. 

Reading and Writing: The individual has a limited understanding of print only through fi-equent 
re-reading; can copy words and phrases and write short sentences. 

Speaking and Listening: The individual can understand fi^equently used words in context and 
very simple phrases spoken slowly and with some repetition; survival needs can be 
communicated simply, and there is some understanding of simple questions. 

Examples of Test Scores: 181 - 200 CASAS; SPL/MELT 2 -4. 

Intermediate ESL 

Functional Skills: The individual can meet basic survival and social needs, can follow some 
simple oral and written instruction and has some ability to understand on the telephone. 

Reading and Writing: The individual can read simple material on familiar subjects, but has 
difficulty with authentic materials; can write simple paragraphs on siuvival topics and personal 
issues with some error. 

Speaking and Listening: The individual can understand simple learned phrases and new phrases 
containing familiar vocabulary; can converse on familiar topics beyond survival needs; can 
clarify speech through rewording and asking questions. There is use and understanding of basic 
grammar. 

Examples of Test Scores: 201 -220 CASAS; SPL/MELT 5-6. 

Advanced ESL 

Functional Skills: The individual can understand general conversations, participate effectively in 
familiar situations, satisfy routine survival and social needs and follow oral and written 
instructions. Individuals also can understand conversation containing some unfamiliar 
vocabulary on many everyday subjects, but may need repetition, rewording, or slower ^eech. 
Reading and Writing: The individual can read materials on abstract topics and descriptions and 
narrations of factual material. The individual can write descriptions and short essays and can 
complete complex forms and applications. There is a general ability to use English effectively to 
meet most routine social and work situations. 

Speaking and Listening: The individual can converse with no or minimal difficulty in 
conversation, can communicate over the telephone on familiar subjects and has basic control of 
grammar; understands descriptive and spoken narrative and can comprehend abstract concepts in 
familiar contexts. 

Examples of Test Scores: 221 and above CASAS; SPL/MELT 7-10. 
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TRAINING PLANS AND RECOMMENDATIONS 



Training on the criteria for student data reporting will occur in September 1997. The 
training will be offered via an interactive teleconference, which will be broadcast to locations 
coordinated through the Professional Development Centers. 

The three-hour teleconference will train programs to implement the criteria through a 
han(k-on demonstration and interactive discussion. The intended participants will be program 
admimstrators and, if ^plicable, support staff who are responsible for collecting and reporting 
student data, and who are resp)onsible for training their program staff to effectively apply the criteria 
to the student data collection process. Thus, one to two representatives fix>m each ABLE-fimded 
program will participate in the teleconference. 

Each PDC will arrange downlink site(s) in their respective regions to enable programs to 
participate as a group. If PDC coordinators so choose, they may coordinate multiple viewing sites 
in a region in order to accommodate travel and space restrictions. PDC staff will attend and 
facilitate or assist in facilitation of the training at each site. 

The teleconference will be organized and broadcast by the Western Adult Literacy Resource 
Center. The Resource Center will perform the following: / 

1 . Arrange for broadcast date and time and establish broadcast coordinates. 

2. Inform by a mailing all ABLE-funded programs and PDCs of the teleconference, providing 
information on the date, time, and coordinates. 

3. Provide techmcal assistance to downlink sites to assure reception of the broadcast. 

4. Produce the teleconference, consulting with ABLE and project staff on the training content. 

5. Produce video copies of the teleconference for distribution to the PDCs and AdvancE for 
viewing at a later date. 

The Center for Literacy and ABLE staff will perform the following: 

1 . Plan the content of the teleconference. 

2. Coordinate production of the teleconference, including scripting, with the Western Resource 
Center (this will involve rehearsal and development of facilitation guidelines. 

3. Present the teleconference. 
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Select one: 

1 . Completed the category at 
which enrolled and left 
the program. 


Description 


Exit database 


Student ID: unique internal student ID number 
assigned for this student. 


Site ID: The site from which the student is being 
exited. 


Exit ID: The unique internal ID number for this 
record. 


Exit Date: The date on which the student exited. 


Contact hours: Total number of hours of instruction 
in which the student participated. 
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GED test taken; results not received. 


Learned the English language (for participants 
whose primary language is not English). 


Entered another education/training program. 


Received U.S. Citizenship. 


Registered to vote for the first time. 


Received driver's license as a result of the program. 


Referred to agencies (other than educational) for 
needed services. 


Obtained a job. 


Obtained a better job or salary, or secured job 
retention. 


Was removed from public assistance. 


Met personal objective. 
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Pre-test math score (if applicable). 


Post-test date. 
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Select one: Male or Female 


Select one: 

1 . American Indian or 
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